When setting up an experiment there are three things that are interacting: the experimental unit; be it a subject, a participant, an organic life form, etc, that is being affected by an explanatory variable, also known as a factor. What is being measured is the response variable. The experimenter needs to control as much extraneous influence on the experimental unit as possible to get as pure an affect by the factor as possible. The factor can be applied at different levels to measure the effect at different levels as well. When a control group is used this can be considered another level of the factor where the subjects get no treatment. A control group gives a point of reference for the other treatment levels AND also can help in creating a blind study. Blinding will counteract people's subjectivity, their reactions to subtle cues, during an experiment. A single-blind or double-blind study means that one or two, repectively, of the following is made unaware of the treatment that is being administered to the subject: a) subjects, treatment administrators, b) result evaluators.
There are several methods we employ to keep these extraneous influences at bay. One of these is to randomize the assignment of experimental units to the factor that is to be applied. This allows the use of statistical methods and it decreases the bias caused by uncontrolled sources both known and unknown. By randomizing we spread out the uncontrolled affects across all the treatment groups and levels. It is important to realize that experiments rarely use random sampling. The randomization is used for the purpose of assigning subjects to treatment groups, NOT for representing a population as in survey sampling. Another important design element is the replication of the treatment on a number of subjects. This is especially important if the subjects are not a representative sample of the population. Replication of the entire experiment is further validation of its credibility and can expand the representation to different or larger populations. This replication of the entire experiment is especially important if the previous experiments do not represent a population. When a small group or single experimental unit is used it provides anecdotal results that can assist in subsequent experimental designs. I think case studies are one example of this.
Sometimes experimental units have attributes that we can't control. These can be addressed by using blocks. This insures that the attributes are equally spread across the groups. This design method is not required, but applied when needed. Blocking is a way to isolate an attribute that is variable between groups. By putting each group in a block the affect of the attribute can be more clearly seen. In the Pew Study, even though it wasn't an experiment with a treatment, blocking was a way to isolate the people with landlines from the people with cell phones only. Once a group is "blocked" the randomization of treatment is applied within the block. This is called randomized block design. it is sort of like running two parallel experiments, one on each block that you form. In a retrospective or prospective study Matching subjects can be used, much like blocking, to reduce variation. A subject in each group is matched with similar characteristics. Blocking is to experiments as stratifying is to sampling.
Multi-factor experiments apply more than one factor in different combinations to each group. Here is where the diagrams are most important. It is sort of like setting up a tournament to be sure that every combination possible is covered.
Confounding, lurking,
Deveaux explained that diagrams are widely used in experimental design.And boxplots are a good way to begin comparing treatments of the groups. The Pew Study and blocks for bias.
Statistical significance?
Lisa, Good discussion on experiments and blocks. Develope the ideas of confounding and lurking variables. How do they differ? Which of these result in "spurious correlations"?
When setting up an experiment there are three things that are interacting: the experimental unit; be it a subject, a participant, an organic life form, etc, that is being affected by an explanatory variable, also known as a factor. What is being measured is the response variable. The experimenter needs to control as much extraneous influence on the experimental unit as possible to get as pure an affect by the factor as possible. The factor can be applied at different levels to measure the effect at different levels as well. When a control group is used this can be considered another level of the factor where the subjects get no treatment. A control group gives a point of reference for the other treatment levels AND also can help in creating a blind study. Blinding will counteract people's subjectivity, their reactions to subtle cues, during an experiment. A single-blind or double-blind study means that one or two, repectively, of the following is made unaware of the treatment that is being administered to the subject: a) subjects, treatment administrators, b) result evaluators.
There are several methods we employ to keep these extraneous influences at bay. One of these is to randomize the assignment of experimental units to the factor that is to be applied. This allows the use of statistical methods and it decreases the bias caused by uncontrolled sources both known and unknown. By randomizing we spread out the uncontrolled affects across all the treatment groups and levels. It is important to realize that experiments rarely use random sampling. The randomization is used for the purpose of assigning subjects to treatment groups, NOT for representing a population as in survey sampling. Another important design element is the replication of the treatment on a number of subjects. This is especially important if the subjects are not a representative sample of the population. Replication of the entire experiment is further validation of its credibility and can expand the representation to different or larger populations. This replication of the entire experiment is especially important if the previous experiments do not represent a population. When a small group or single experimental unit is used it provides anecdotal results that can assist in subsequent experimental designs. I think case studies are one example of this.
Sometimes experimental units have attributes that we can't control. These can be addressed by using blocks. This insures that the attributes are equally spread across the groups. This design method is not required, but applied when needed. Blocking is a way to isolate an attribute that is variable between groups. By putting each group in a block the affect of the attribute can be more clearly seen. In the Pew Study, even though it wasn't an experiment with a treatment, blocking was a way to isolate the people with landlines from the people with cell phones only. Once a group is "blocked" the randomization of treatment is applied within the block. This is called randomized block design. it is sort of like running two parallel experiments, one on each block that you form. In a retrospective or prospective study Matching subjects can be used, much like blocking, to reduce variation. A subject in each group is matched with similar characteristics. Blocking is to experiments as stratifying is to sampling.
Multi-factor experiments apply more than one factor in different combinations to each group. Here is where the diagrams are most important. It is sort of like setting up a tournament to be sure that every combination possible is covered.
Confounding, lurking,
Deveaux explained that diagrams are widely used in experimental design.And boxplots are a good way to begin comparing treatments of the groups.
The Pew Study and blocks for bias.
Statistical significance?
Lisa, Good discussion on experiments and blocks. Develope the ideas of confounding and lurking variables. How do they differ? Which of these result in "spurious correlations"?